Skip to content

Enhance controlling manually braccio #73

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 23 commits into from
May 12, 2022

Conversation

91volt
Copy link
Collaborator

@91volt 91volt commented May 3, 2022

Enhanced version of the manually-controlling-braccio example:

  • Added "PINCH" joints control (braccio grip)
  • Switch between joints is now triggered directly by the enter button or joystick click
  • A label at the center of the screen shows the current selected joints
  • The UI buttons now behave as push buttons, the checked state is triggered while joystick directional inputs are pressed, and revert to default state on joystick release.
  • Removal of the mainMenu screen
  • UI buttons availability change accordingly to the selected joints

Note: since the driver definition and hardware buttons mapping happens inside the library and that only the "KEY_ENTER" lvgl input triggers the LV_EVENT_PRESSED/PRESSING/RELEASED, is not possible to achieve a "push button" behavior without adding a routine in loop that check if Braccio.getKey() has changed to zero in order to remove the checked state.
The definition of another input driver for this example where each input is identified as KEY_ENTER would make possible to exploit the states of lvgl with all the physical inputs.

@CLAassistant
Copy link

CLAassistant commented May 3, 2022

CLA assistant check
All committers have signed the CLA.

@github-actions
Copy link

github-actions bot commented May 3, 2022

Memory usage change @ ffd88d7

Board flash % RAM for global variables %
arduino:mbed_nano:nanorp2040connect 0 - 0 0.0 - 0.0 0 - 0 0.0 - 0.0
Click for full report table
Board examples/Braccio_Learn_and_Repeat
flash
% examples/Braccio_Learn_and_Repeat
RAM for global variables
% examples/Controlling_Manually_Braccio
flash
% examples/Controlling_Manually_Braccio
RAM for global variables
% examples/Platform_Tutorials/lessons/01-programming-the-braccio-display/01_creating_a_button
flash
% examples/Platform_Tutorials/lessons/01-programming-the-braccio-display/01_creating_a_button
RAM for global variables
% examples/Platform_Tutorials/lessons/01-programming-the-braccio-display/02_designing_the_button
flash
% examples/Platform_Tutorials/lessons/01-programming-the-braccio-display/02_designing_the_button
RAM for global variables
% examples/Platform_Tutorials/lessons/01-programming-the-braccio-display/03_creating_a_menu
flash
% examples/Platform_Tutorials/lessons/01-programming-the-braccio-display/03_creating_a_menu
RAM for global variables
% examples/Platform_Tutorials/lessons/01-programming-the-braccio-display/04_testing_it_out
flash
% examples/Platform_Tutorials/lessons/01-programming-the-braccio-display/04_testing_it_out
RAM for global variables
% examples/Platform_Tutorials/lessons/01-programming-the-braccio-display/05_display_challenge
flash
% examples/Platform_Tutorials/lessons/01-programming-the-braccio-display/05_display_challenge
RAM for global variables
% examples/Platform_Tutorials/lessons/02-navigatting-the-display-menu/01_playing_with_the_Joystick
flash
% examples/Platform_Tutorials/lessons/02-navigatting-the-display-menu/01_playing_with_the_Joystick
RAM for global variables
% examples/Platform_Tutorials/lessons/02-navigatting-the-display-menu/02_handling_events_in_the_menu
flash
% examples/Platform_Tutorials/lessons/02-navigatting-the-display-menu/02_handling_events_in_the_menu
RAM for global variables
% examples/Platform_Tutorials/lessons/02-navigatting-the-display-menu/03_navigate_challenge_I
flash
% examples/Platform_Tutorials/lessons/02-navigatting-the-display-menu/03_navigate_challenge_I
RAM for global variables
% examples/Platform_Tutorials/lessons/02-navigatting-the-display-menu/04_navigate_challenge_II
flash
% examples/Platform_Tutorials/lessons/02-navigatting-the-display-menu/04_navigate_challenge_II
RAM for global variables
% examples/Platform_Tutorials/lessons/03-playing-with-the-motors/01_playing_with_the_motors
flash
% examples/Platform_Tutorials/lessons/03-playing-with-the-motors/01_playing_with_the_motors
RAM for global variables
% examples/Platform_Tutorials/lessons/03-playing-with-the-motors/02_selecting_the_motor_with_the_enter_button
flash
% examples/Platform_Tutorials/lessons/03-playing-with-the-motors/02_selecting_the_motor_with_the_enter_button
RAM for global variables
% examples/Platform_Tutorials/lessons/03-playing-with-the-motors/03_moving_the_motors_with_the_joystick
flash
% examples/Platform_Tutorials/lessons/03-playing-with-the-motors/03_moving_the_motors_with_the_joystick
RAM for global variables
% examples/Platform_Tutorials/lessons/03-playing-with-the-motors/04_servo_motors_challenge
flash
% examples/Platform_Tutorials/lessons/03-playing-with-the-motors/04_servo_motors_challenge
RAM for global variables
% examples/Platform_Tutorials/lessons/04-integration-of-previous-learnings/01_playing_with_a_joint_angle_gauge
flash
% examples/Platform_Tutorials/lessons/04-integration-of-previous-learnings/01_playing_with_a_joint_angle_gauge
RAM for global variables
% examples/Platform_Tutorials/lessons/04-integration-of-previous-learnings/02_selecting_the_motor_in_the_LCD_menu
flash
% examples/Platform_Tutorials/lessons/04-integration-of-previous-learnings/02_selecting_the_motor_in_the_LCD_menu
RAM for global variables
% examples/Platform_Tutorials/lessons/04-integration-of-previous-learnings/03_learnings_challenge_I
flash
% examples/Platform_Tutorials/lessons/04-integration-of-previous-learnings/03_learnings_challenge_I
RAM for global variables
% examples/Platform_Tutorials/lessons/04-integration-of-previous-learnings/04_learnings_challenge_II
flash
% examples/Platform_Tutorials/lessons/04-integration-of-previous-learnings/04_learnings_challenge_II
RAM for global variables
% examples/Platform_Tutorials/projects/p01-moving-braccio/01_aligning_braccio
flash
% examples/Platform_Tutorials/projects/p01-moving-braccio/01_aligning_braccio
RAM for global variables
% examples/Platform_Tutorials/projects/p01-moving-braccio/02_waving_with_Braccio
flash
% examples/Platform_Tutorials/projects/p01-moving-braccio/02_waving_with_Braccio
RAM for global variables
% examples/Platform_Tutorials/projects/p01-moving-braccio/03_moving_challenge
flash
% examples/Platform_Tutorials/projects/p01-moving-braccio/03_moving_challenge
RAM for global variables
% examples/Platform_Tutorials/projects/p02-controlling-braccio-manually/01_controlling_manually_Braccio
flash
% examples/Platform_Tutorials/projects/p02-controlling-braccio-manually/01_controlling_manually_Braccio
RAM for global variables
% examples/Platform_Tutorials/projects/p02-controlling-braccio-manually/02_manual_control_challenge
flash
% examples/Platform_Tutorials/projects/p02-controlling-braccio-manually/02_manual_control_challenge
RAM for global variables
% examples/Platform_Tutorials/projects/p03-learning-mode/01_Braccio_learning_mode
flash
% examples/Platform_Tutorials/projects/p03-learning-mode/01_Braccio_learning_mode
RAM for global variables
% examples/Platform_Tutorials/projects/p03-learning-mode/02_learning_challenge
flash
% examples/Platform_Tutorials/projects/p03-learning-mode/02_learning_challenge
RAM for global variables
% examples/Tools/Braccio_Basic
flash
% examples/Tools/Braccio_Basic
RAM for global variables
% examples/Tools/Braccio_LearnByDoing
flash
% examples/Tools/Braccio_LearnByDoing
RAM for global variables
% examples/Tools/Factory_Set_ID_Interactive
flash
% examples/Tools/Factory_Set_ID_Interactive
RAM for global variables
% examples/Tools/Factory_Set_Initial_Servo_Position
flash
% examples/Tools/Factory_Set_Initial_Servo_Position
RAM for global variables
% examples/Tools/LCD_Custom_Menu
flash
% examples/Tools/LCD_Custom_Menu
RAM for global variables
% examples/Tools/LCD_Menu_Joystick
flash
% examples/Tools/LCD_Menu_Joystick
RAM for global variables
% examples/Tools/LCD_Motors
flash
% examples/Tools/LCD_Motors
RAM for global variables
% examples/Tools/Test_Inputs
flash
% examples/Tools/Test_Inputs
RAM for global variables
% examples/Tools/Test_Motor_Angular_Control
flash
% examples/Tools/Test_Motor_Angular_Control
RAM for global variables
% examples/Tools/Test_Motor_Communication
flash
% examples/Tools/Test_Motor_Communication
RAM for global variables
%
arduino:mbed_nano:nanorp2040connect 0 0.0 0 0.0 N/A N/A N/A N/A 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0
Click for full report CSV
Board,examples/Braccio_Learn_and_Repeat<br>flash,%,examples/Braccio_Learn_and_Repeat<br>RAM for global variables,%,examples/Controlling_Manually_Braccio<br>flash,%,examples/Controlling_Manually_Braccio<br>RAM for global variables,%,examples/Platform_Tutorials/lessons/01-programming-the-braccio-display/01_creating_a_button<br>flash,%,examples/Platform_Tutorials/lessons/01-programming-the-braccio-display/01_creating_a_button<br>RAM for global variables,%,examples/Platform_Tutorials/lessons/01-programming-the-braccio-display/02_designing_the_button<br>flash,%,examples/Platform_Tutorials/lessons/01-programming-the-braccio-display/02_designing_the_button<br>RAM for global variables,%,examples/Platform_Tutorials/lessons/01-programming-the-braccio-display/03_creating_a_menu<br>flash,%,examples/Platform_Tutorials/lessons/01-programming-the-braccio-display/03_creating_a_menu<br>RAM for global variables,%,examples/Platform_Tutorials/lessons/01-programming-the-braccio-display/04_testing_it_out<br>flash,%,examples/Platform_Tutorials/lessons/01-programming-the-braccio-display/04_testing_it_out<br>RAM for global variables,%,examples/Platform_Tutorials/lessons/01-programming-the-braccio-display/05_display_challenge<br>flash,%,examples/Platform_Tutorials/lessons/01-programming-the-braccio-display/05_display_challenge<br>RAM for global variables,%,examples/Platform_Tutorials/lessons/02-navigatting-the-display-menu/01_playing_with_the_Joystick<br>flash,%,examples/Platform_Tutorials/lessons/02-navigatting-the-display-menu/01_playing_with_the_Joystick<br>RAM for global variables,%,examples/Platform_Tutorials/lessons/02-navigatting-the-display-menu/02_handling_events_in_the_menu<br>flash,%,examples/Platform_Tutorials/lessons/02-navigatting-the-display-menu/02_handling_events_in_the_menu<br>RAM for global variables,%,examples/Platform_Tutorials/lessons/02-navigatting-the-display-menu/03_navigate_challenge_I<br>flash,%,examples/Platform_Tutorials/lessons/02-navigatting-the-display-menu/03_navigate_challenge_I<br>RAM for global variables,%,examples/Platform_Tutorials/lessons/02-navigatting-the-display-menu/04_navigate_challenge_II<br>flash,%,examples/Platform_Tutorials/lessons/02-navigatting-the-display-menu/04_navigate_challenge_II<br>RAM for global variables,%,examples/Platform_Tutorials/lessons/03-playing-with-the-motors/01_playing_with_the_motors<br>flash,%,examples/Platform_Tutorials/lessons/03-playing-with-the-motors/01_playing_with_the_motors<br>RAM for global variables,%,examples/Platform_Tutorials/lessons/03-playing-with-the-motors/02_selecting_the_motor_with_the_enter_button<br>flash,%,examples/Platform_Tutorials/lessons/03-playing-with-the-motors/02_selecting_the_motor_with_the_enter_button<br>RAM for global variables,%,examples/Platform_Tutorials/lessons/03-playing-with-the-motors/03_moving_the_motors_with_the_joystick<br>flash,%,examples/Platform_Tutorials/lessons/03-playing-with-the-motors/03_moving_the_motors_with_the_joystick<br>RAM for global variables,%,examples/Platform_Tutorials/lessons/03-playing-with-the-motors/04_servo_motors_challenge<br>flash,%,examples/Platform_Tutorials/lessons/03-playing-with-the-motors/04_servo_motors_challenge<br>RAM for global variables,%,examples/Platform_Tutorials/lessons/04-integration-of-previous-learnings/01_playing_with_a_joint_angle_gauge<br>flash,%,examples/Platform_Tutorials/lessons/04-integration-of-previous-learnings/01_playing_with_a_joint_angle_gauge<br>RAM for global variables,%,examples/Platform_Tutorials/lessons/04-integration-of-previous-learnings/02_selecting_the_motor_in_the_LCD_menu<br>flash,%,examples/Platform_Tutorials/lessons/04-integration-of-previous-learnings/02_selecting_the_motor_in_the_LCD_menu<br>RAM for global variables,%,examples/Platform_Tutorials/lessons/04-integration-of-previous-learnings/03_learnings_challenge_I<br>flash,%,examples/Platform_Tutorials/lessons/04-integration-of-previous-learnings/03_learnings_challenge_I<br>RAM for global variables,%,examples/Platform_Tutorials/lessons/04-integration-of-previous-learnings/04_learnings_challenge_II<br>flash,%,examples/Platform_Tutorials/lessons/04-integration-of-previous-learnings/04_learnings_challenge_II<br>RAM for global variables,%,examples/Platform_Tutorials/projects/p01-moving-braccio/01_aligning_braccio<br>flash,%,examples/Platform_Tutorials/projects/p01-moving-braccio/01_aligning_braccio<br>RAM for global variables,%,examples/Platform_Tutorials/projects/p01-moving-braccio/02_waving_with_Braccio<br>flash,%,examples/Platform_Tutorials/projects/p01-moving-braccio/02_waving_with_Braccio<br>RAM for global variables,%,examples/Platform_Tutorials/projects/p01-moving-braccio/03_moving_challenge<br>flash,%,examples/Platform_Tutorials/projects/p01-moving-braccio/03_moving_challenge<br>RAM for global variables,%,examples/Platform_Tutorials/projects/p02-controlling-braccio-manually/01_controlling_manually_Braccio<br>flash,%,examples/Platform_Tutorials/projects/p02-controlling-braccio-manually/01_controlling_manually_Braccio<br>RAM for global variables,%,examples/Platform_Tutorials/projects/p02-controlling-braccio-manually/02_manual_control_challenge<br>flash,%,examples/Platform_Tutorials/projects/p02-controlling-braccio-manually/02_manual_control_challenge<br>RAM for global variables,%,examples/Platform_Tutorials/projects/p03-learning-mode/01_Braccio_learning_mode<br>flash,%,examples/Platform_Tutorials/projects/p03-learning-mode/01_Braccio_learning_mode<br>RAM for global variables,%,examples/Platform_Tutorials/projects/p03-learning-mode/02_learning_challenge<br>flash,%,examples/Platform_Tutorials/projects/p03-learning-mode/02_learning_challenge<br>RAM for global variables,%,examples/Tools/Braccio_Basic<br>flash,%,examples/Tools/Braccio_Basic<br>RAM for global variables,%,examples/Tools/Braccio_LearnByDoing<br>flash,%,examples/Tools/Braccio_LearnByDoing<br>RAM for global variables,%,examples/Tools/Factory_Set_ID_Interactive<br>flash,%,examples/Tools/Factory_Set_ID_Interactive<br>RAM for global variables,%,examples/Tools/Factory_Set_Initial_Servo_Position<br>flash,%,examples/Tools/Factory_Set_Initial_Servo_Position<br>RAM for global variables,%,examples/Tools/LCD_Custom_Menu<br>flash,%,examples/Tools/LCD_Custom_Menu<br>RAM for global variables,%,examples/Tools/LCD_Menu_Joystick<br>flash,%,examples/Tools/LCD_Menu_Joystick<br>RAM for global variables,%,examples/Tools/LCD_Motors<br>flash,%,examples/Tools/LCD_Motors<br>RAM for global variables,%,examples/Tools/Test_Inputs<br>flash,%,examples/Tools/Test_Inputs<br>RAM for global variables,%,examples/Tools/Test_Motor_Angular_Control<br>flash,%,examples/Tools/Test_Motor_Angular_Control<br>RAM for global variables,%,examples/Tools/Test_Motor_Communication<br>flash,%,examples/Tools/Test_Motor_Communication<br>RAM for global variables,%
arduino:mbed_nano:nanorp2040connect,0,0.0,0,0.0,N/A,N/A,N/A,N/A,0,0.0,0,0.0,0,0.0,0,0.0,0,0.0,0,0.0,0,0.0,0,0.0,0,0.0,0,0.0,0,0.0,0,0.0,0,0.0,0,0.0,0,0.0,0,0.0,0,0.0,0,0.0,0,0.0,0,0.0,0,0.0,0,0.0,0,0.0,0,0.0,0,0.0,0,0.0,0,0.0,0,0.0,0,0.0,0,0.0,0,0.0,0,0.0,0,0.0,0,0.0,0,0.0,0,0.0,0,0.0,0,0.0,0,0.0,0,0.0,0,0.0,0,0.0,0,0.0,0,0.0,0,0.0,0,0.0,0,0.0,0,0.0,0,0.0,0,0.0,0,0.0,0,0.0,0,0.0,0,0.0,0,0.0,0,0.0,0,0.0,0,0.0,0,0.0,0,0.0,0,0.0,0,0.0,0,0.0,0,0.0,0,0.0,0,0.0,0,0.0,0,0.0

@clodpheasant clodpheasant requested review from aentinger and Esquirio May 3, 2022 07:44
@Esquirio
Copy link
Collaborator

Esquirio commented May 3, 2022

I updated the code with the Braccio.lvgl_lock/unlock statements and commented on it. The code is good, but it is slow to respond to the commands with the joystick. That is something that we don't see in the project 02 code.

With this new example, we will have two similar codes (Controlling_Manually_Braccio.ino and 01_controlling_manually_Braccio.ino). In this case, don't recommend making the codes identical because of the target audience of the content.

Copy link
Collaborator

@Esquirio Esquirio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code is okay, but please check the message in the thread.

@91volt
Copy link
Collaborator Author

91volt commented May 3, 2022

The code is good, but it is slow to respond to the commands with the joystick. That is something that we don't see in the project 02 code.

I couldn't test it with the braccio attached, tomorrow morning I'm going to investigate on it. It might be due to the delay inside the if statement of the loop.

Is a scope creep of this PR - but - a refactoring in the library on how the lvgl driver functions are exposed would be helpful in order to allow to customize the KEY events paired with the physical buttons. This slowness would be surely overcome if all input buttons could be read as KEY_ENTER events in this sketch, making possible to use the LV_EVENT_PRESSING and LV_EVENT_RELEASED inside the event handler when left,right,up,down joystick buttons are used.
Discussion regarding this behavior of lvgl KEYs: lvgl forum link

@Esquirio
Copy link
Collaborator

Esquirio commented May 4, 2022

After you test it, you could try to create multiple screens for each layout (shoulder, elbow, wrist, and gripper), or you can delete the button matrix object and recreate it again with the new design. I think one of these updates should fix the problem.

@aentinger aentinger force-pushed the enhance_controlling_manually_braccio branch from 414975a to 5ff8dde Compare May 9, 2022 05:13
@aentinger
Copy link
Contributor

After you test it, you could try to create multiple screens for each layout (shoulder, elbow, wrist, and gripper), or you can delete the button matrix object and recreate it again with the new design. I think one of these updates should fix the problem.

Good input, I'll take it into considerations.

@aentinger
Copy link
Contributor

Hi @91volt @Esquirio ☕ 👋

I did a fair bit of rework on the example. Please test it and let me know what you think.

I'm experiencing a bit of jerkiness of the arm which is caused due to
a) mechanical tolerance of the servo motors when holding zero position and
b) our way of controlling the servo by providing target angles to be reached. If we provide a too small angle increment the servo does not move at all. If we provide a too big angle increment it moves in starts and stops (i.e. jerky movement). Feel free to fine-tune angle increments and run time.

CC @clodpheasant

@Esquirio
Copy link
Collaborator

This code is the most stable of all. I ran the code many times, and it worked most of the time correctly. Now, the response of the movements when we move the joystick is quick.

Once or twice, I had to restart the Arduino because the movement of the Elbow and the wrist didn't work well, but I couldn't figure out why it was necessary.

@aentinger aentinger merged commit 3a63895 into main May 12, 2022
@aentinger aentinger deleted the enhance_controlling_manually_braccio branch May 12, 2022 13:40
@per1234 per1234 added topic: documentation Related to documentation for the project type: enhancement Proposed improvement labels May 12, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
topic: documentation Related to documentation for the project type: enhancement Proposed improvement
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants